-
Notifications
You must be signed in to change notification settings - Fork 1.1k
feat: show timeseries [order by timseries] clause #17065
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
Conversation
…subtree of metadata tree. initial version. # Conflicts: # iotdb-core/datanode/src/main/java/org/apache/iotdb/db/schemaengine/schemaregion/mtree/impl/mem/MTreeBelowSGMemoryImpl.java
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull request overview
This PR adds ORDER BY TIMESERIES [ASC|DESC] support to SHOW TIMESERIES and introduces an in-memory metadata-tree optimization to speed up large OFFSET values by pruning whole subtrees using cached subtree measurement counts.
Changes:
- Extends SQL grammar + statement model to represent
ORDER BY TIMESERIES(ASC/DESC). - Updates planning/execution to support ordering (global sort in multi-region; ordered traversal pushdown in single region) and optimizes
OFFSETusing subtree measurement statistics. - Adds/updates schema-region read plan plumbing and introduces an integration test for ordering behavior.
Reviewed changes
Copilot reviewed 24 out of 24 changed files in this pull request and generated 6 comments.
Show a summary per file
| File | Description |
|---|---|
| iotdb-core/antlr/src/main/antlr4/org/apache/iotdb/db/qp/sql/IoTDBSqlParser.g4 | Adds orderByTimeseriesClause to SHOW TIMESERIES grammar. |
| iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/plan/parser/ASTVisitor.java | Parses ORDER BY TIMESERIES and enforces semantic constraints. |
| iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/plan/statement/metadata/ShowTimeSeriesStatement.java | Stores order-by-timeseries flags and ordering direction. |
| iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/plan/statement/metadata/ShowStatement.java | Adds getLimitWithOffset() helper for multi-region limit/offset handling. |
| iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/plan/planner/LogicalPlanVisitor.java | Wires ordering and limit/offset handling into logical plan; adds global SortNode when needed. |
| iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/plan/planner/LogicalPlanBuilder.java | Extends schema source planning to pass ordering flags; adjusts limit handling. |
| iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/plan/planner/plan/node/metadata/read/TimeSeriesSchemaScanNode.java | Persists order-by-timeseries flags through plan-node serialization. |
| iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/plan/planner/OperatorTreeGenerator.java | Passes ordering flags down into schema source creation. |
| iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/plan/planner/distribution/SimpleFragmentParallelPlanner.java | Ensures type provider generation for order-by-timeseries scenarios. |
| iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/execution/operator/schema/source/SchemaSourceFactory.java | Threads ordering flags into TimeSeriesSchemaSource creation. |
| iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/execution/operator/schema/source/TimeSeriesSchemaSource.java | Threads ordering flags into schema-region read plans. |
| iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/execution/operator/schema/source/LogicalViewSchemaSource.java | Updates show-timeseries plan invocation signature. |
| iotdb-core/datanode/src/main/java/org/apache/iotdb/db/schemaengine/schemaregion/read/req/IShowTimeSeriesPlan.java | Adds region-level order-by-timeseries flags. |
| iotdb-core/datanode/src/main/java/org/apache/iotdb/db/schemaengine/schemaregion/read/req/SchemaRegionReadPlanFactory.java | Extends plan factory API for ordering flags. |
| iotdb-core/datanode/src/main/java/org/apache/iotdb/db/schemaengine/schemaregion/read/req/impl/ShowTimeSeriesPlanImpl.java | Implements the new ordering-flag accessors and equality updates. |
| iotdb-core/datanode/src/main/java/org/apache/iotdb/db/schemaengine/schemaregion/mtree/impl/mem/mnode/IMemMNode.java | Adds subtree measurement count getters/setters to mem-nodes. |
| iotdb-core/datanode/src/main/java/org/apache/iotdb/db/schemaengine/schemaregion/mtree/impl/mem/mnode/basic/BasicMNode.java | Adds cached subtreeMeasurementCount field and size estimation adjustment. |
| iotdb-core/datanode/src/main/java/org/apache/iotdb/db/schemaengine/schemaregion/mtree/impl/mem/mnode/impl/{AboveDatabaseMNode,DatabaseMNode,MeasurementMNode}.java | Delegates subtree measurement count storage to BasicMNode. |
| iotdb-core/datanode/src/main/java/org/apache/iotdb/db/schemaengine/schemaregion/mtree/impl/mem/MTreeBelowSGMemoryImpl.java | Maintains subtree counts, rebuilds on snapshot load, prunes traversal for OFFSET, and sorts children for ordered traversal. |
| iotdb-core/datanode/src/test/java/org/apache/iotdb/db/queryengine/execution/operator/schema/SchemaQueryScanOperatorTest.java | Updates test wiring for new plan parameters. |
| iotdb-core/datanode/src/test/java/org/apache/iotdb/db/metadata/schemaRegion/SchemaRegionTestUtil.java | Updates test plan factory calls with new params. |
| integration-test/src/test/java/org/apache/iotdb/db/it/schema/IoTDBShowTimeseriesOrderByTimeseriesIT.java | Adds IT coverage for ordering + offset/limit and conflict semantics. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
...ava/org/apache/iotdb/db/schemaengine/schemaregion/mtree/impl/mem/MTreeBelowSGMemoryImpl.java
Show resolved
Hide resolved
...ava/org/apache/iotdb/db/schemaengine/schemaregion/mtree/impl/mem/MTreeBelowSGMemoryImpl.java
Show resolved
Hide resolved
...ava/org/apache/iotdb/db/schemaengine/schemaregion/mtree/impl/mem/MTreeBelowSGMemoryImpl.java
Show resolved
Hide resolved
...ava/org/apache/iotdb/db/schemaengine/schemaregion/mtree/impl/mem/mnode/basic/BasicMNode.java
Outdated
Show resolved
Hide resolved
...ode/src/main/java/org/apache/iotdb/db/queryengine/plan/statement/metadata/ShowStatement.java
Show resolved
Hide resolved
...test/src/test/java/org/apache/iotdb/db/it/schema/IoTDBShowTimeseriesOrderByTimeseriesIT.java
Show resolved
Hide resolved
…chemaFilter() is non-null and when subtree may be not completely included in pathPattern.
…plate is extended
feat: show timeseries [order by timseries] clause
Support
ORDER BY TIMESERIESinSHOW TIMESERIESand OptimizeOFFSETwith Subtree Measurement StatisticsBrief
This PR introduces ordering support for
SHOW TIMESERIESby measurement name (lexicographical) and optimizes the performance of theOFFSETclause using subtree measurement statistics.Currently,
SHOW TIMESERIESsupports theORDER BY TIMESERIESclause.When the query scope is within a single database,
OFFSET Xno longer requires traversing the first X measurements one by one. Instead, it skips subtrees in the metadata tree based on subtree measurement counts.Example:
DESC: descending orderASC: ascending order (default if not specified)Design & Implementation Details
1. Syntax Extension
SHOW TIMESERIESto supportorderByTimeseriesClause.ShowTimeSeriesStatementto record sorting requirements.2. Logical Plan Construction
SortNodeat the top of the logical plan to enable global ordering across partitions.LIMITandOFFSETare pushed down to the metadata tree traversal stage inLogicalPlanVisitor.3. Limit/Offset Handling in Multi-Partition Queries
OFFSETcannot be pushed down.Instead,
limit' = limit + offsetis used.when
limit = 0(which is treated as unlimited in IoTDB),limit'was incorrectly pushed down, causing unexpected truncation of results.4. Ordered Traversal in Metadata Tree
MTreeBelowSGMemoryImpl, the iteration strategy of child nodes inschemaReaderis overridden under ordering mode.5. Subtree Measurement Count
Implemented subtree measurement statistics (only for in-memory metadata tree):
subtreeMeasurementCounttoIMemMNodeand related classes.subtreeMeasurementCountalong ancestor paths during measurement insertion and deletion.During schemaReader construction,
acceptFullMatchedNodeis overridden to skip subtree traversal whensubtreeMeasurementCount < offset.Tests
Functional Tests
IoTDBShowTimeseriesOrderByTimeseriesITto verify correctness of ordering and offset behavior.Performance Evaluation
Xranges from 0 to 900,000 with a step of 5,000.Impact
SHOW TIMESERIES.OFFSETvalues in single-partition scenarios